1. Field of the Invention
The present invention relates to a technique of correcting image blurring of a captured image arising from a shake of an image capturing apparatus by using a method of deforming an image in the image capturing apparatus.
2. Description of the Related Art
In recent years, an image stabilization function has become popular along with the development of a technique of correcting a shake acting on an image capturing apparatus. The image stabilization function corrects not only image blurring of a captured image caused by a camera shake when a user is in a still state, but also image blurring of a captured image generated in shooting while the user is walking. When the user shoots an image while walking, the following image blurring is generated in the captured image, in addition to image blurring of the captured image in the horizontal and vertical directions. Examples are image blurring in which a captured image rotates owing to rotation of the image capturing apparatus about the optical axis, and image blurring in which a captured image is distorted into a trapezoidal shape owing to a tilt of the image capturing apparatus with respect to an object.
As a method of correcting various kinds of image blurring generated in a captured image in shooting while walking, there is known a method of calculating the image deformation amount of a captured image and deforming the image to cancel the image deformation amount (see Japanese Patent Laid-Open No. 2011-029735).
Since rotational image blurring about the optical axis and trapezoidal distortion image blurring caused by a perspective greatly stand out on the wide angle side but are hardly conspicuous on the telephoto-side, the effect of correction is little. For this reason, there is also proposed a method of permitting correction of rotational image blurring on the wide angle side with respect to the boundary of a given focal length and inhibiting it on the telephoto-side (see Japanese Patent Laid-Open No. 2006-071743).
In the technique disclosed in Japanese Patent Laid-Open No. 2011-029735, the amount of deformation generated in a captured image owing to a shake acting on the image capturing apparatus is decomposed into deformation components of translation (horizontal/vertical), perspective (horizontal/vertical), enlargement/reduction, rotation, and shear. Filtering processing and the like are then performed for the respective deformation components, and a projective transformation matrix (homography matrix) is calculated. However, this method has the following problems.
More specifically, in the technique disclosed in Japanese Patent Laid-Open No. 2011-029735, since a correction amount is calculated for each deformation component mentioned above, the calculation amount becomes very large and processing becomes complicated. When a shake is large, if all the deformation components are corrected, a finally outputtable image range may become narrow, resulting in poor image quality.
For example, FIG. 11 shows an example of image stabilization by image deformation when the image capturing apparatus rotates about the optical axis. An outer solid-line portion in FIG. 11 indicates a captured image. To correct rotation about the optical axis, the captured image is rotated about an image center O. The rotation of the captured image about the image center O results in a dotted-line image in FIG. 11. At this time, if an output image has the same range as that of the captured image, the image has no data at the four corners. To prevent this, the output image range becomes a hatched region in FIG. 11. Although correction of rotation about the optical axis has been exemplified in FIG. 11, it is necessary to ensure, for each of the remaining deformation components, an image region (to be defined as extra pixels hereinafter) which cannot be output for deformation component. For this reason, the image range becomes narrow, and it becomes difficult to maintain the quality of video after performing image stabilization.
Especially when horizontal and vertical translation image blurs of a captured image are generated by a camera shake, the amount of motion on the image becomes larger as the focal length becomes larger. To capture an unblurred image by correcting the horizontal and vertical translation image blurs, a larger correction amount is required on the telephoto-side than on the wide angle side, and a larger number of extra pixels need to be ensured for that.
A technique disclosed in Japanese Patent Laid-Open No. 2006-071743 considers only still image shooting and does not consider moving image shooting, as in a video camera. Thus, the following problems arise.
More specifically, in a method of switching, with respect to the boundary of a given focal length, whether to correct rotational image blurring, rotational image blurring is corrected on the wide angle side with respect to the boundary of this focal length and is not corrected on the telephoto-side. When zooming is performed in moving image shooting, rotational image blurring suddenly appears with respect to the boundary of the focal length.
Further, when a zoom key is arranged to operate it in a direction perpendicular to the optical axis, the zooming operation readily causes rotational image blurring.